Improving Scalability of Evolutionary Robotics with Reformulation

Creating systems that can operate autonomously in complex environments is a challenge for contemporary engineering techniques. Automatic design methods offer a promising alternative, but so far they have not been able to produce agents that outperform manual designs. One such method is evolutionary robotics. It has been shown to be a robust and versatile tool for designing robots to perform simple tasks, but more challenging tasks at present remain out of reach of the method. In this thesis I discuss and attack some problems underlying the scalability issues associated with the method. I present a new technique for evolving modular networks. I show that the performance of modularity-biased evolution depends heavily on the morphology of the robot’s body and present a new method for co-evolving morphology and modular control. To be able to reason about the new technique I develop reformulation framework: a general way to describe and reason about metaoptimization approaches. Within this framework I describe a new heuristic for developing metaoptimization approaches that is based on the technique for co-evolving morphology and modularity. I validate the framework by applying it to a practical task of zero-g autonomous assembly of structures with a fleet of small robots. Although this work focuses on the evolutionary robotics, methods and approaches developed within it can be applied to optimization problems in any domain

Morphological modularity can enable the evolution of robot behavior to scale linearly

In evolutionary robotics, populations of robots are typically trained in simulation before one or more of them are instantiated as physical robots. However, in order to evolve robust behavior, each robot must be evaluated in multiple environments.If an environment is characterized by $f$ free parameters, each of which can take one of $n_p$ features, each robot must be evaluated in all $n_p^f$ environments to ensure robustness.Here we show that, if the robots are constrained to have modular morphologies and controllers, they only need to be evaluated in $n_p$ environments to reach the same level of robustness.This becomes possible because the robots evolve such that each module of the morphology allows the controller to independently recognize a familiar percept in the environment, and each percept corresponds to one of the environmental free parameters. When exposed to a new environment, the robot perceives it as a novel combination of familiar percepts which it can solve without requiring further training. A non-modular morphology and controller however perceives the same environment as a completely novel environment, requiring further training.This acceleration in evolvability -- the rate of the evolution of adaptive and robust behavior -- suggests that evolutionary robotics may become a scalable approach for automatically creating complex autonomous machines, if the evolution of neural and morphological modularity is taken into account

Modularity and Sparsity: Evolution of Neural Net Controllers in Physically Embodied Robots

While modularity is thought to be central for the evolution of complexity and evolvability, it remains unclear how systems boot-strap themselves into modularity from random or fully integrated starting conditions. Clune et al. (2013) suggested that a positive correlation between sparsity and modularity is the prime cause of this transition. We sought to test the generality of this modularity-sparsity hypothesis by testing it for the first time in physically embodied robots. A population of ten Tadros — autonomous, surface-swimming robots propelled by a flapping tail — was used. Individuals varied only in the structure of their neural net control, a 2 x 6 x 2 network with recurrence in the hidden layer. Each of the 60 possible connections was coded in the genome, and could achieve one of three states: -1, 0, 1. Inputs were two light-dependent resistors and outputs were two motor control variables to the flapping tail, one for the frequency of the flapping and the other for the turning offset. Each Tadro was tested separately in a circular tank lit by a single overhead light source. Fitness was the amount of light gathered by a vertically oriented sensor that was disconnected from the controller net. Reproduction was asexual, with the top performer cloned and then all individuals entered into a roulette wheel selection process, with genomes mutated to create the offspring. The starting population of networks was randomly generated. Over ten generations, the population’s mean fitness increased two-fold. This evolution occurred in spite of an unintentional integer overflow problem in recurrent nodes in the hidden layer that caused outputs to oscillate. Our investigation of the oscillatory behavior showed that the mutual information of inputs and outputs was sufficient for the reactive behaviors observed. While we had predicted that both modularity and sparsity would follow the same trend as fitness, neither did so. Instead, selection gradients within each generation showed that selection directly targeted sparsity of the connections to the motor outputs. Modularity, while not directly targeted, was correlated with sparsity, and hence was an indirect target of selection, its evolution a by-product of its correlation with sparsity